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ABSTRACT 

While statistical procedures are well known for comparing 
hierarchically related (nested) covariance structure models, statistical 
tests for comparing nonhierarchically related (nonnested) models have proven 
more elusive. While isolated attempts have been made, none exists within the 
commonly used maximum likelihood estimation framework, thereby compromising 
these methods' accessibility and general applicability. The current work 
builds on a distance measure originally proposed by C. Rao (1945; 1949), and 
its application to distances between covariance structure models (A. Kumar 
and S. Sharma, 1999), thereby proposing a method for conducting a statistical 
test of such distances in order to assess formally the distinctness between 
models— nested or nonnested. An illustration is presented, and simulation 
evidence is provided to validate the performance of the proposed method. Two 
appendixes contain an illustration of the model for data generation and a 
program to compute distances between covariance matrices. (Contains 1 table, 

2 figures, and 28 references.) (Author/SLD) 
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Abstract 

While statistical procedures are well-known for comparing hierarchically related (nested) 
covariance structure models, statistical tests for comparing non-hierarchically related (nonnested) 
models have proven more elusive. While isolated attempts have been made, none exists within 
the commonly-used maximum likelihood estimation framework, thereby compromising these 
methods' accessibility and general applicability. The current work builds upon a distance 
measure originally proposed by Rao (1945; 1949), and its application to distances between 
covariance structure models (Kumar & Sharma, 1999), thereby proposing a method for 
conducting a statistical test of such distances in order to assess formally the distinctness between 
models - nested or noimested. An illustration is presented, and simulation evidence is provided 
to validate the performance of the proposed method. 
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A Statistical Test For Comparing Nonnested Covariance Structure Models 

The comparison of competing models is a keen interest for researchers interested in 
structural equation models and related techniques, be they path analysis, confirmatory factor 
analysis (CFA), or other members of the structural equation modeling (SEM) family. A number 
of indices have been developed for evaluating models individually and are often employed in 
comparing alternative models. Model comparisons involving statistical tests have generally been 
limited to likelihood ratio tests difference tests) for models in a hierarchically related 
system. We define hierarchically related systems as consisting of at least one nested and one 
general model such that the nested model is a special case of the general model. If Ma is defined 
by a vector of parameters 0 ^ with degrees of freedom and Mb is defined by a 

vector of parameters 0 g = , . . . , J with Vq degrees of freedom then Mb is nested within Ma 

when 0gis a subset of 0^ , i.e., <m^. 

While the aim of most goodness-of-fit statistical tests is to evaluate the null hypothesis 
that one particular model fits the data, often a more pragmatic concern is whether or not two 
models approximate the imderlying population equally well (Golden, 2000). Along those lines, 
Kumar and Sharma (1999) argued that in acknowledging: (1) all models are approximations, (2) 
multiple models might fit the data identically, and (3) it is sometimes necessary to respecify 
existing models, researchers are led to questions of discrepancy among competing models. 
Investigating the relation(s) among competing models directly might offer valuable evidence for 
assessing model discrepancy, ultimately as an aide in model selection. The current work is not 
as concerned with the discrepancies between individual models and the data as with the 
discrepancy between the models themselves. Further, the approach we take is to assess the 
discrepancy between models directly, rather than indirectly via a comparison of their respective 
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discrepancies to a fixed point, i.e., the sample data. Comparing models that fit the data equally 
well via the indirect strategy will not necessarily reveal any discrepancy between the models; a 
strategy of direct comparison will. This important distinction is visually represented in Figure 1. 
In the center of the diagram is the data to which Models M\, M 2 , and M 3 are fit. The dashed lines 
between the data and the models represent data-model (mis)fit. The collinearity between the 
data, Ml, and M 3 reflects that M 3 is nested within Mi. In hierarchical systems, the distinction 
between comparing models directly or indirectly via reference to the individual models’ fit to the 
data is moot. The discrepancy between Mi and M 3 is completely captured by the difference 
between the data-model fit of each model; here, difference tests (of individual data-model 
fit) serve to assess the discrepancy between models. This is not necessarily the case in 
nonhierarchical systems. Mi and M 2 fit the data equally well; the discrepancy between them (the 
solid line) cannot be assessed using only the models’ respective data-model fit. We therdfore 
aim at a procedure that statistically tests the discrepancy between competing models directly. 
Establishing statistical that tests of discrepancy for non-hierarchically related models has proven 
most challenging, though procedures aimed at this problem have been proposed. Still, no 
dominant paradigm exists for the comparison of nonnested models within the maximum 
likelihood estimation (MLE) framework. 

In the next section of this paper, strategies for model comparison in various research 
settings are reviewed. Next, a distance measure for evaluating the differences between models 
originally proposed by Rao (1945; 1949) and first applied by Kumar and Sharma (1999) to 
distances between covariance structure models will be described. An empirical strategy for 
conducting a statistical test of such distances will be explored, the purpose being to derive a 
general test for models within the SEM framework that provides a statistical assessment of the 
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The difference between the values for hierarchically related models M 3 and Mi, 

= x\ - X\ > treated as a with degrees of freedom equal to the difference between 

the number of free parameters in each model, A v' = v'j - v', . Under a true null hypothesis stating 
correct and equivalent data-model fit, follows a central x^ distribution with Ai^ ; if the null 
hypothesis is false, Aj^ follows the appropriate noncentral distribution. Such techniques 
are also employed in evaluating model fit individually with respect to the observed data (i.e., not 
in a comparative setting as is the main concern of this paper). That is, any individual model can 
be thought of as a constrained version of a saturated model, i.e., a model that fits perfectly and 
has no degrees of freedom; thus the statistical test of the model x^ can be thought of as a test 

between a nested model and a general (saturated) model. 

When models are not hierarchically related, the likelihood ratio test is no longer valid, 
and thus one typically resorts to a comparison of existing fit indices. As noted by several authors 
(e.g., Bagozzi & Yi, 1990; Byrne & Goffm, 1993; Widaman, 1985), differences in fit indices 
(absolute, parsimonious, and/or incremental) can provide a useful practical heuristic for 
assessing differences in data-model fit. However, these do not provide a formal statistical test. 
The Akaike Information Criterion, AIC (Akaike, 1987), is also commonly recommended for 
facilitating a choice among models, nested or nonnested. In short, the index allows for a 
determination of the relative replicability of the models under comparison, without providing a 
direct assessment of their overall adequacy of data-model fit. The AIC is a parsimonious fit 
index, in which absolute data-model fit is penalized for model complexity (i.e., having more 
parameters). Though common, such a strategy is not without criticism. First, as noted by 
Golden (2000) and Mulaik (2001), with increasing sample size such complexity penalties have 
less influence on the index (for a detailed discussion of this issue and subsequent criticisms of 
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the AIC, see Mulaik, 2001). There exist alternate information criteria, such as the Schwarz 
Information Criterion (SIC; Schwarz, 1978) and the Consistent AIC (CAIC; Bozdogan, 1987), 
which incorporate sample size into the penalty term. However such adjustments do not 
overcome the second and, for the purposes of this work, the most important shortcoming of this 
approach, namely that comparison of fit indices or information criteria does not constitute a 
formal statistical test of the difference in data-model fit of competing models, nested or 
nonnested. 

To such end, attempts have been made on a number of fronts for statistically comparing 
nonnested models. Work has progressed in combining the psychometric literature on estimating 
latent variables with the econometric literature on testing noimested models estimated with 
instrumental variables. Oczkowski and Farrell (1998) implemented a two-stage least squares 
(2SLS) estimator derived from Bollen (1996) and conduct a Cox-type J-test and an 
encompassing F-test derived from Smith (1992) to compare models employing different 
measures of market orientation. Unfortunately, as Oczkowski and Farrell (1998) noted, 
simulations suggest the J-test tends to over-reject the tme null hypothesis while the F-test tends 
to possess less power when the null is false. 

Other approaches in the comparison of nonnested models have come from employing 
Bayesian analyses. Most Bayesian methods of comparing nonnested models provide posterior 
probabilities that are roughly interpretable as probabilities of model correctness, but only if the 
correct model is part of the specified set. A Bayesian Cross Validated Likelihood method 
(BCVL) originally proposed by Rust and Schmittlein (1985) has been shown to overcome this 
qualification, though if the correct model is not among those being compared, posteriors can only 
be treated as conditional probabilities (Fomell & Rust, 1989). The BCVL method consists of 
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specifying a prior probability of correctness for each competing model, estimating each model 
from half of the observed data and calculating posterior probabilities for each model by 
combining (via Bayes theorem) the prior probability for the model and the likelihood of 
observing the second half of the data. There have been several extensions of the BCVL since it 
was first proposed, notably generalizations to allow the comparison of nonnested models 
(Fomell & Rust, 1989) and the comparison of nonnested SEM models that may have unique 
observable indicators (Rust et al., 1995) 

As potentially useful as the BCVL method and its extensions may be for model selection, 
there is no statistical test for it to compare models. To conduct a pseudo-hypothesis test. Rust 
and Schmittlein (1985) advocate setting priors for each model to mimic a hypothesis test. In 
comparing hierarchically related models (akin to a classical test at the .05 significance level), the 
nested model would have a prior of .95 and the rejection of the null hypothesis of equivalent 
data-model fit would be Justified when the posterior probability for the general model exceeds .5 
(Rust & Schmittlein, 1985; for a clear illustration of the relationship between the prior and model 
selection, see Fomell & Rust, 1989). However appropriate this approach may be for determining 
which model fits better, which was the concern of Rust and Schmittlein (1985) and Fomell and 
Rust (1989), it reiterates the difference between the focus of this work and others. It is possible 
for the pseudo-hypothesis test to indicate that two models do not differ in data-model fit, even 
when the models are different. This may occur because, like other methods that involve the 
likelihood, the BCVL indirectly compares the models by virtue of assessing the discrepancy of 
each to a fixed point, the sample data. When the likelihood of the data under each model is the 
same (e.g., M\ and Mi in Figure 1), comparing the model values or computing the BCVL will 
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not be able to distinguish between models. The method proposed in this paper is aimed at being 
able to reveal model difference (when present) even when the models fit the data equally well. 

Along the lines of a more classical approach, Golden (2000) extends the work of Vuong 
( 1 989), describing a large sample procedure designed to test if the estimated data-model fit for 
one model is significantly different from that of another model. Golden (2000) demonstrates this 
test on logistic regression models rather than covariance structure models, and while the general 
nature of the test suggests its applicability to other analyses warrants investigation, the procedure 
(1) is tied to residuals of observations, which are not the unit of analysis in covariance structure 
analyses, and (2) involves the assessing the discrepancy between models indirectly via their 
discrepancy to the data. 

The development of statistical tests of nonnested models has been drawing attention from 
varying perspectives, as there is no standard paradigm for conducting such tests of noimested 
models in SEM and related techniques. It is to that end that this enterprise is directed, namely, 
the derivation of a test of covariance structure models that is applicable under very general 
conditions. This alternative and potentially more accessible line of reasoning draws upon a 
distance measure originally proposed by Rao (1945; 1949). Following Kumar and Sharma's 
(1999) application of Rao's Distance to describe the distance between two covariance structure 
models, our work proposes a procedure for conducting hypothesis tests of covariance structure 
models that is applicable under general conditions. 

Rao’s Distance Measure 

For two p-variate normal distributions, each has p means and p{p+\)!2 variances and 
covariances. The well-known Mahalanobis distance. 
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-Ji2) 



( 3 ) 



(Mahalanobis, 1936), assesses the disparity between the two distributions' pxl mean vectors 
(centroids fi, and fi^) alone, thereby assuming homogeneity of covariance matrices. Rao (1945; 
1949) extended this work to create a distance measure able to assess distributional disparity 
across all p(p+3)/2 first and second moments. If one wishes to assume identical mean vectors 
(fi, = Pi ). ^ simplified version of Rao's Distance ( RD ) may be used to assess the disparity 
between two p-variate distributions' covariance matrices. As shown by Atkinson and Mitchell 



(1981), 



RD(S„5:j 




( 4 ) 



where L, and Lj ^he covariance matrices, and 2, are the eigenvalues that satisfy the 

equation 

|L, -2,.L,| = 0 (5) 

As noted by Kumar and Sharma (1999), if L, and are positive definite and symmetric, all the 
p eigenvalues will be strictly positive. 

RD has several desirable characteristics. As discussed by Kumar and Sharma (1999), it 
satisfies standard properties of distance such as symmetry with respect to its arguments (i.e., 

RD (L , , L 2 ) = RD (L 2 , L , ) ) and invariance to changes in variables' scale when applied to both 
populations. In addition, as the reader may easily verify, RD is always nonnegative and the 
distance from any population to itself is zero. 

Kumar and Sharma (1999) suggest the descriptive application of RD to the model-implied 
covariance matrices arising from competing models imposed upon the same sample covariance 
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matrix. The disparity between two models can be assessed using RD(L^,'L 2 ) where 2, and 
are model-implied covariance matrices. The focus of this paper is to extend the use of RD as a 
comparative index by incorporating a statistical test of significance. 

Rao (1945) provided a framework for conducting tests of statistical significance for the 
metric. In the case where one population is specified and the other is represented only by a 
sample, an estimate of the distance between the populations is defined as the distance between 
the set of maximum likelihood parameter estimates based on the sample and the set of 
parameters that define the specified population. Rao (1945) also provided a large sample 
approximation of the variance of the estimated distance. If 0„ is a vector of parameters defining 

the null population and 0 is a vector of the maximum likelihood estimates obtained from a 
sample, then the statistic 

w = — (6) 

[f(j?2)(0,0„))]^ 

is asymptotically distributed as a standard normal variate (Rao, 1945). The extension to the case 
of two independent samples case is conceptually straightforward. RD is calculated between the 
two sets of maximum likelihood estimates and the standard error term is adjusted to reflect the 
variability associated with estimating the distance from two independent samples (Rao, 1945). 

In terms of covariance structure modeling, the derivation of a sampling distribution for the 
distance between independent sample covariance matrices from the same population would rest 
(in part) on the distribution of sample covariance matrices. It is well-known that the variances 
and covariances obtained from randomly selecting data from a /?-variate normally distributed 
population follow a Wishart distribution (Wishart, 1928). In the case of independent covariance 
matrices, it is possible that an estimate of the sampling distribution could be derived. Unlike 
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such a scenario, however, model-implied covariance matrices fit to a common sample covariance 
matrix are not independent. The analytical derivation of a standard error for RD requires (in 
part) the cross-group covariances of model-implied variances and covariances. Such derivations 
are generally imwieldy as well as model-dependent, thus making a derivation of the theoretical 
sampling distribution and the hypothesis testing it facilitates practically intractable. The current 
paper therefore offers an empirical approach to the estimation of the sampling distribution in 
question, thereby facilitating hypothesis testing of the disparity between models, nested or 
nonnested, applied to the same data. 



An Empirical Approach 

Consider Model 1 with m i parameters contained in set 0 , = (^i i , • • • , ) and Model 2 with 

mi parameters contained in set Gj = )• Model 1 and Model 2 may or may not be 

correct in truth; the test proposed concerns the distinction between them regardless of their 

correctness. Next, consider a Model C with me parameters contained in set 

0c = )= )^(^ 2 i>--->^ 2 ;., )• This model, referred to as the "Child" model, 

contains the complete set of parameters common to both models. 

For any population covariance matrix fitting Model C, Model 1 and Model 2 also fit 
perfectly. In terms of E,) = Ej) = itD(E,, Ej) = 0, where E^. refers to 

the covariance matrix for the model. Turning to samples, for any sample covariance matrix S 
drawn from a population that fits Model C, Model 1 and Model 2 will fit better, but only due to 

chance. RD{1L^, E,) and RD{Z^, Ej) fluctuate above 0 only randomly, and thus 

i?Z)(E| , Ej ) also fluctuates randomly above 0. It is the distribution of this latter term, i.e., the 
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distribution of RD between model-implied covariance matrices of nonnested models under the 
null condition of equivalent population fit, which serves as the focus of the current work. In 
light of the aforementioned difficulties in analytically deriving the sampling distribution of 

RD{t^,t^) under null conditions, a resampling strategy is employed to approximate this 
distribution and facilitate significance tests. 

For nxp sample data matrix Y with covariance matrix S, the distance between Model 1 
and Model 2 is 



and II, Y and Ilj y are the model-implied covariance matrices obtained from fitting Model 1 and 



against which i?Z)(Il, y, maybe evaluated. 

To that end, the model-implied covariance matrix E,; y is obtained by applying Model C 
to S. The raw data in matrix Y may then be transformed (Bollen & Stine, 1993) to yield 
Z = YS~'^^Ec y , where the covariance matrix of data in Z is now E^ y • For these data Model C 
fits perfectly, as do Models 1 and 2. That is, 

/?Z)(Ecz, E, z) = i?E>(Ecz, 22 2 ) = -^(2i,z> ^ 2 ,z) = 0- For a pseudosample Z* bootstrapped 
from Z, the RD value representing the distance between Model 1 and Model 2 will only differ 




( 7 ) 



where are the /=!,..., eigenvalues that satisfy the equation 




( 8 ) 



Model 2 to the data in Y, respectively.' We now seek to derive a null sampling distribution 



' As there are multiple models fit to data both prior to and after a transformation, a richer notation is required. In 
eqs. (7) and (8) and hereafter, model-implied covariance matrices will be indexed by two subscripts, the first 
denoting the model, the second denoting the data to which the model is fit. 
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from (i.e., be greater than) 0 randomly. Upon repeated bootstrap resampling from Z, a sampling 
distribution for these RD values can be derived empirically. This distribution is an estimate of 

sampling variability imder the null condition that , Xlj) = 0 , and thus serves as the frame 

of reference for evaluating the observed y, tj.y) from the original raw data in Y. 

Specifically, for a desired a-level, if the observed iJD(2,y, Xljy) exceeds the (l-a)xl00“’ 

percentile value in the empirical sampling distribution, then the null hypothesis of model 
equivalence is rejected and the inference is made that Model 1 and Model 2 are different. 

Illustration 

Let Model G be the full model depicted in Figure 2; let Model 1 be the constrained 
version of Model G such that the cross loadings, and (the dashed paths) are constrained 

to be 0. Let Model 2 be the constrained version of Model G such that and (the bold bi- 
directional factor covariances) are 0. Model 1 and Model 2 are therefore nonnested models; the 
discrepancy between them (with respect to a common set of data) is of interest, and it is such a 
discrepancy that stands in need of formal statistical test. 

A multivariate normal data set Y of size n = 200 cases was generated based upon Model 
G where the loadings ( A ) for the measured variables and the covariance matrices for the factors 
(O) and errors (0 ) are given in Appendix A. Data was generated in GAUSS (Aptech Systems, 
1996) by using Vale and Maurelli’s (1983) extension of the method originally proposed by 
Fleishman (1978). Specifically, the program NNORMULT (Nevitt & Hancock, 1999) was 
used. Both Model 1 and Model 2 were fit to this data using EQS (Bentler, 1998), yielding 
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model-implied covariance matrices £, y and £ 2 ,v Computation of 
performed in GAUSS.^ 

The Child Model containing the complete set of parameters common to both models is 
the constrained version of Model G where both sets of constraints have been imposed; 

, and (Z >32 are all constrained to equal 0. Model C was fit to the data, yielding model-implied 
covariance matrix y . The raw data matrix Y was then transformed to yield Z = YS , 

where S is the covariance matrix of the raw data Y, resulting in transformed data matrix Z with 
covariance matrix E^ y • 

A total of 200 bootstrapped pseudosamples, k= 1,..., 200, each of size 200 were 
obtained by resampling with replacement from Z. For each bootstrapped sample. Model 1 and 
Model 2 were fit to the data andi?Z)(i:, was calculated. The resulting empirical 



distribution of RD(Z, reflects the (sampling) variability in RD(L^, £ 2 ) under the 



null condition of no discrepancy between models. If the observed RD (£, y , £ 2 ,y ) (7)) 

exceeds the (1 - cr) x 1 00'*’ percentile value in this empirical sampling distribution, then the 
discrepancy between Model 1 and Model 2 is statistically significant. 

In order to assess the performance of the bootstrap and the possible relationship between 
statistical tests of RD and in the case of nested models. Model C was fit to each 
bootstrapped pseudosample. Let 



r* =(«-i)(in: 



y.z* 



-\n\zl\ + tr[ZlZ-;^.]-p) 



( 9 ) 



^ A GAUSS (Aptech Systems, 1996) program for computing RD is given in Appendix B 
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be a test statistic associated with fitting the model to /7-variate bootstrapped data Z] . The 
difference between such test statistics for hierarchically related models (e.g., ^ - T, ^ 

and ATi. = J approximates Fitting Model C to the bootstrapped data in addition 

to fitting Model 1 and Model 2 facilitates an investigation of the relationship between R£> and 
Az^ in the case of nested models (i.e., Model C is nested within both Model 1 and Model 2). 

For each bootstrapped pseudosample, both RD and AT were calculated for the distance between 
Model C and Model 1 and for the distance between Model C and Model 2. To evaluate the 
extent of the agreement between the proposed test of RD and the Az^ test, the process of data 
generation, fitting, transformation, bootstrapping, and fitting was repeated 100 times. 



Results 

Summary results from the calculation of the 95'*’ percentile over the 100 replications of 
the empirical null distribution of RD(t,, tj) are given in Table 1, though in the absence of an 
analytical expectation, they are neither easily interpretable nor of interest. In practice, the 
empirical 95* percentile from one set of bootstrapped samples would serve as our criterion for 
testing RD(E, V , tj.v) the .05 level. What is of interest is the relationship between the 
proposed test of RD and the Az^ test in the cases of hierarchically related models (i.e.. Model 1 

^ We hesitate to formally treat any Tj (and, by extension, any AT ) as statistics for several reasons (Bollen & 
Stine, 1993). First, the distribution of Tj (and AT) to Z^ approximate result. Second, while the data- 
generating population is continuous, the bootstrap distribution is necessarily discrete. Third, to treat Tj (or AT ) as 

a Z^ requires an assumption of multivariate normality, which, even if met in the data-generating population, does 
not necessarily hold in transformed data that is to be bootstrapped. Following Bollen and Stine (1993), we 
acknowledge these limitations and proceed to treat Tj and AT as approximations to the model Z > 

respectively. 
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to Model C and Model 2 to Model C); if (the proposed test of) RD is operating in the same way 
as (the test of) , there ought to be a monotonic relationship between the empirical null 
sampling distributions of RD and AT . Inspection of plots for both hierarchical systems 
indicated a nonlinear monotonic relationship between RD and AT . To assess the degree of 
agreement between RD and AT , Spearman’s rank-order correlation coefficient (rj) was 
computed between RD and AT for the 200 bootstrapped values within each replication, for each 
of the two nested comparisons. Over 100 replications, the average values of rs for the 
comparison between Models 1 and 2 to Model C were 0.998323 and 0.999477, respectively, and 
the worst values of r$ for the comparison between Models 1 and 2 to Model C were .993817 and 
.998764, respectively (Table 1). These values indicates a high degree of monotonicity and 
strongly support our contention that that the result of the proposed test of RD would be 
consistent with the result of the test. That is, these results imply that the behavior of RD in 
the case of sampling from a true null condition almost perfectly mimics that of Ihe case 

of nested models. 



Discussion 

The concern of this investigation is the performance of the proposed procedure for testing 
RD for competing covariance structure models. In the case of nested models, the test of t^x^ 
the do minan t paradigm within the framework of MLE of SEM models. It is argued here that the 
level of agreement between the proposed test of RD and the A;jf ^ test is excellent. The 
observed Spearman rank-order correlations indicate a high degree of monotonicity between the 
distributions of RD and the h.x ^ ; we have strong evidence to believe that, for any significance 
level, the results of the proposed test of RD (i.e., reject or retain the null hypothesis of no model 
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discrepancy) would be just that of the test. In the case of nested models, the proposed test 
of RD meets the standard of consistency with the ^ test. There is no such standard for 
evaluating the performance of the empirical distribution of RD in the case of nonnested models. 
Its justification rests on appropriateness of employing RD in the analysis of covariance structure 
models (Kumar & Sharma, 1999) and the logic of the procedure presented here. If the logic of 
the process is sound, the proposed procedure ought to mimic well-known procedures in the case 
of nested models. Simulation evidence presented above suggests that this is indeed so. Further, 
the method proposed does not rest on distributional assumptions of the metric; the distribution is 
empirically generated, allowing the extension to the case of comparing nonnested models. 

Conclusion 

Though work has progressed on several fronts in the statistical analysis of nonnested 
covariance structure models, there is currently no standard procedure for the testing of 
differences between such models. A distance measure conceived by Rao (1945; 1949), extended 
by Atkinson and Mitchell (1981) and previously applied to covariance analysis (Kumar & 
Sharma, 1999) was described. This work builds upon the application of this measure to 
covariance structure modeling; we propose a method for conducting a statistical test of the 
measure in order to formally assess the distinctness between models - nested or nonnested. An 
empirical approach for the assessment of the discrepancy between the fit of competing Models 1 
and 2 was described and is summarized as: 

(1) For sample data matrix Y with covariance matrix S, fit Model 1 and Model 2 to yield 
model-implied covariance matrices £, y and 2^ y 

(2) Compute RD(2, y, 22 ,y) 

(3) Define the Child Model as 0c = )= ) 
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(4) Fit the Child Model to Y to yield model-implied covariance matrix • 

(5) Transform the data (Bollen & Stine, 1993) to yield Z = YS"'^^LJ;^y 

(6) ForJt=l,...,200 

a. Bootstrap the pseudosample from Z, Z] 

b. Fit Model 1 and Model 2 to Z\, yielding and 

c. Compute RD (Sj > ^2 z; ^ 

(7) If the observed RD(t^y, tj.v) from (2) exceeds the (l-a)xl00‘'’ percentile value 
from those compiled from (6), reject the null hypothesis of model equivalence. 

An illustration designed to mimic applied analyses was embedded in a simulation designed to 
assess the approach. The results of these investigations are quite promising; the proposed 
procedure is in accordance with existing procedures for the comparison of nested models. It is 
argued that employing the procedure in the comparison of nonnested models is justified on 
logical grounds. 

Further research is currently being conducted to investigate the proposed procedure. In 
the context of the relationship between RD and in the case of nested models, areas of 
interest are (1) the association between p-values obtained from the test of RD and the test of 
AZ^ and (2) the analytical relationship between RD and Az^ ■ Other avenues of pursuit include 
investigating the impact of both sample size and the number of bootstraps, and extensions to 
multiple model comparisons. The Child Model has been defined as the model with the set of 
parameters common to the competing models; another line of research might include the 
viability of other definitions, e.g., a subset of the parameters common to the competing models. 
Further, the discussion of RD presented here concerns models in which mean vectors are 
assumed equal. An equation for RD has been derived in the case of dissimilar mean vectors 
(Atkinson & Mitchell, 1981); the method proposed here might then be applied to the comparison 
of mean structured models with augmented moment matrices (Sdrbom, 1974). 
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Figure 1 : Dashed lines from models M\, Mi, and M 3 to the data indicate data-model (mis)fit. The 
solid line indicates the discrepancy between M\ to M 3 , which cannot be assessed by indirect 
comparisons of the models’ relations to the data. 
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Figure 2; Model G is the model with all parameters present; Model 1 constrains and (the 
dashed paths) to be 0; Model 2 constrains (j)^^ and (the bold bi-directional factor covariances) 
to be 0; Model C imposes the constraints of both Model 1 and Model 2. 
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Measure 


Mean 


Standard 

Deviation 


Minimum 


Maximum 


^ 51,0 


0.998323 


0.000959 


0.993817 


0.999433 


f'si.c 


0.999477 


0.000176 


0.998764 


0.999719 


95“’ %ile 


0.203446 


0.012445 


0.175220 


0.241889 



Table 1 : Model Comparison of nested and nonnested models 
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Appendix A 

Instantiation of Model G for data generation, defined in terms of the matrices of loadings ( A ), 
factor variances and covariances ( C> ), and error variances and covariances ( 0 ). 
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Appendix B 

GAUSS program to compute Rao’s Distance between covariance matrices ‘a’ and ‘b’. 
new; 

a={1.0 0.1, 

0.1 1.0}; 
b={1.0 0.2, 

0.2 1 . 0 }; 
c=a*inv(b); 
evunsort=eig(c); 
ev=rev(sortc(evunsort, 1)); 
lo=ln(ev); 
mult=lo'*lo; 
f=.5*mult; 
rd=SQRT(f); 

PRINT "Distance =" rd; 
end; 



It should be noted that there are other ways to compute RD . For example, Kumar and Sharma 
(1999) include a program based on computing eigenvectors. The program detailed here is used 
because it (1) is more in line with eqs. (4) and (5), and (2) is not based on eigenvectors, which 
are not unique, the use of which may result in different software packages producing different 
values for RD . 
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